You'll be giving up .4 liters of displacement which will adversely affect your torque production and turbo spoolup. Both of which will make you a little bit slower.
But don't let a little reality get in between you and your fantasy, have fun with it.